SIMSBIOHUB-466: Add Observation Measurement Columns #1213

NickPhura · 2024-02-14T23:34:17Z

Links to Jira Tickets

SIMSBIOHUB-466

Description of Changes

Many updates to the Observation Table components and related components

Support for adding, removing, hiding, and populating measurement columns.
New database tables + related service/repo

When creating an observation record, a new subcount record is also created.

Addressed a handful of dependency array warnings and "update a component while rendering another" warnings

Some misc package updates + related fixes.

Pending Critterbase updates

Wire up the stubbed calls to Critterbase, once its new endpoints are ready.

TODOs for another ticket

Update the telemetry datagrid components/tables to use the latest updates from the survey observations components.

Fix the 3-4 app tests that started to fail (probably a side effect of package updates?) I tried fixed them but was making no progress, so they are skipped for now. They are all related to trying to capture select box clicks, which is historically tricky.

Testing Notes

Observations table works as expected, when adding and removing observations with and without measurement columns.

- Manage column components (initial structure) - Autocomplete with stubbed response (issue: input clearing on option select) - Not wired up to sims backend yet - Not wired up to observations table yet

…to SIMSBIOHUB-466

…umns to measurement columns.

…to SIMSBIOHUB-466

codecov-commenter · 2024-02-26T21:23:19Z

Codecov Report

Attention: Patch coverage is 33.71429% with 116 lines in your changes are missing coverage. Please review.

Project coverage is 76.02%. Comparing base (a7cdac3) to head (1e11eda).

Files	Patch %	Lines
api/src/middleware/critterbase-proxy.ts	0.00%	28 Missing ⚠️
api/src/services/observation-service.ts	22.85%	27 Missing ⚠️
api/src/repositories/subcount-repository.ts	25.00%	24 Missing ⚠️
api/src/services/subcount-service.ts	26.66%	11 Missing ⚠️
api/src/services/platform-service.ts	0.00%	8 Missing ⚠️
api/src/database/db.ts	25.00%	6 Missing ⚠️
api/src/app.ts	0.00%	5 Missing ⚠️
api/src/services/critterbase-service.ts	66.66%	5 Missing ⚠️
...rojectId}/survey/{surveyId}/observations/delete.ts	0.00%	1 Missing ⚠️
api/src/services/bctw-service.ts	0.00%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##              dev    #1213       +/-   ##
===========================================
+ Coverage   56.26%   76.02%   +19.75%     
===========================================
  Files         576      235      -341     
  Lines       17647     9629     -8018     
  Branches     2751     1353     -1398     
===========================================
- Hits         9929     7320     -2609     
+ Misses       7056     1896     -5160     
+ Partials      662      413      -249

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

al-rosenthal

Ran into a few issues when testing this locally:
Validation Banner doesn't show

Hide required columns (like the sample sites)
Add new row and fill
Save (see error dialog)
No banner

Hiding newly added columns

Add new measurement
New column cannot be toggled
Close/ Reopen measurements panel
Newly added measurement is still cannot be toggled

Deleting Columns from measurement popup

Removing a single measurement from the list seems to work as expected
Removing all added measurements seems to reload the panel and add columns back to list (might be data related)

api/src/paths/project/{projectId}/survey/{surveyId}/observations/index.ts

api/src/repositories/observation-repository.ts

api/src/utils/media/xlsx/xlsx-file.ts

app/src/components/data-grid/taxonomy/TaxonomyDataGridViewCell.tsx

app/src/features/surveys/observations/measurements/list/MeasurementsListCard.tsx

app/src/features/surveys/observations/measurements/search/MeasurementsSearchAutocomplete.tsx

...atures/surveys/observations/observations-table/configure-table/ConfigureColumnsContainer.tsx

app/src/constants/session-storage.ts

NickPhura · 2024-02-27T22:31:10Z

@al-rosenthal

Ran into a few issues when testing this locally: Validation Banner doesn't show

I think this is an existing issue. It seems that we only validate the sampling columns if at least one of them is populated, probably to do with the observation import maybe? The comments in that code talk about not validating sampling columns if its an incidental observation.

Hiding newly added columns

I fixed a few issues with the visibility controls. I think it should be happy now.

Deleting Columns from measurement popup

Yeah this is a more interesting issue. The problem is that the measurement data is still part of the row data, and just with the way the useEffect is set up right now, when you remove all measurement columns it re-loads the columns from the existing data, hence those columns showing up again suddenly. There should be a fix, but to date we only ever do row based deletes, so ill have to poke at it.

al-rosenthal

Retested this and saw something else:

Add a new row
Discard Changes
Open Measurement Panel
Cannot add measurements (probably an issue with state)

NickPhura · 2024-02-28T00:42:38Z

@al-rosenthal

Retested this and saw something else:

Should be fixed. Was not resetting the row modes model on discard

app/src/features/surveys/observations/measurements/dialog/MeasurementsDialog.tsx

...atures/surveys/observations/observations-table/configure-table/ConfigureColumnsContainer.tsx

curtisupshall

Looks good to me! 📏

al-rosenthal

🍾

al-rosenthal

⚒️

sonarqubecloud · 2024-02-29T23:45:28Z

Quality Gate failed

Failed conditions
12.0% Duplication on New Code (required ≤ 3%)

See analysis details on SonarCloud

NickPhura and others added 30 commits February 2, 2024 17:54

Initial commit

2c1a559

- Manage column components (initial structure) - Autocomplete with stubbed response (issue: input clearing on option select) - Not wired up to sims backend yet - Not wired up to observations table yet

WIP

5702265

Working autocomplete list

33b697a

Rename/move components, add jsdoc

06e7dc6

Tweaks

09b9149

Add critterbase proxy

8ca8da9

Initial code to add measurement columns to observation table

697d2a9

Fix react warnings

2ccb3a7

Move col defs to separate file

6be290f

Add qual/quan measurement col defs

d85bfbd

Add hasError to observations table context

71a91db

Update apiRefs to use api exposed through datagrid params

2476970

Cleanup measurement type

4ea3008

Merge branch 'dev' into SIMSBIOHUB-466

0fbbe81

Updating Menu for Column Configuration

d75b418

basic version working

7206465

Merge branch 'SIMSBIOHUB-466' of https://github.com/bcgov/biohubbc in…

85ad6af

…to SIMSBIOHUB-466

added new attribute types to front end

9096f1e

UI Updates

7b330f2

Merge branch 'SIMSBIOHUB-466' of https://github.com/bcgov/biohubbc in…

d08129d

…to SIMSBIOHUB-466

Update frontend to start sending measurement data to backend

9c27713

Merge branch 'SIMSBIOHUB-466' of https://github.com/bcgov/biohubbc in…

a3228a7

…to SIMSBIOHUB-466

stubbed out basic critter base call

2852ff3

Merge branch 'SIMSBIOHUB-466' of https://github.com/bcgov/biohubbc in…

50a835b

…to SIMSBIOHUB-466

Fix/patch a bunch of react dependency warnings. Rename additional col…

0ec9d24

…umns to measurement columns.

Updating Table Pagination Options

34147a4

continued work with inserting attributes

bed1944

Merge branch 'SIMSBIOHUB-466' of https://github.com/bcgov/biohubbc in…

34aa89b

…to SIMSBIOHUB-466

remove isLoading from taxonomy context

67360d0

Comment more dependency warnings

d9bbb92

Address some code smells

cbe2440

NickPhura added the Ready For Review PR is ready for review label Feb 26, 2024

al-rosenthal reviewed Feb 27, 2024

View reviewed changes

curtisupshall reviewed Feb 27, 2024

View reviewed changes

Fix observations column visibility toggle

e7517e5

NickPhura added 4 commits February 27, 2024 14:37

code review updates

85308d4

Code review fixes

074afcb

Update endpoint number to integer

f937842

Fix test

0ed68bd

al-rosenthal reviewed Feb 28, 2024

View reviewed changes

NickPhura added 2 commits February 27, 2024 16:35

Add missing pipeline env var

735d9d5

Fix row modes bug

f224b8f

curtisupshall reviewed Feb 29, 2024

View reviewed changes

app/src/features/surveys/observations/measurements/dialog/MeasurementsDialog.tsx Outdated Show resolved Hide resolved

...atures/surveys/observations/observations-table/configure-table/ConfigureColumnsContainer.tsx Outdated Show resolved Hide resolved

NickPhura added 2 commits February 29, 2024 09:34

Code review fixes

e0ab7a3

Merge remote-tracking branch 'origin/dev' into SIMSBIOHUB-466

d1ce3a1

curtisupshall previously approved these changes Feb 29, 2024

View reviewed changes

al-rosenthal previously approved these changes Feb 29, 2024

View reviewed changes

Remove manual rowModes set

63af389

NickPhura dismissed stale reviews from al-rosenthal and curtisupshall via 63af389 February 29, 2024 18:39

fix row modes issue when discarding

8c89ed8

curtisupshall approved these changes Feb 29, 2024

View reviewed changes

al-rosenthal approved these changes Feb 29, 2024

View reviewed changes

Merge branch 'dev' into SIMSBIOHUB-466

1e11eda

NickPhura merged commit d2a1c85 into dev Mar 1, 2024
16 of 17 checks passed

NickPhura deleted the SIMSBIOHUB-466 branch March 1, 2024 00:19

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

SIMSBIOHUB-466: Add Observation Measurement Columns #1213

SIMSBIOHUB-466: Add Observation Measurement Columns #1213

NickPhura commented Feb 14, 2024 •

edited

Loading

codecov-commenter commented Feb 26, 2024 •

edited

Loading

al-rosenthal left a comment •

edited

Loading

NickPhura commented Feb 27, 2024

al-rosenthal left a comment

NickPhura commented Feb 28, 2024

curtisupshall left a comment

al-rosenthal left a comment

al-rosenthal left a comment

sonarqubecloud bot commented Feb 29, 2024

SIMSBIOHUB-466: Add Observation Measurement Columns #1213

SIMSBIOHUB-466: Add Observation Measurement Columns #1213

Conversation

NickPhura commented Feb 14, 2024 • edited Loading

Links to Jira Tickets

Description of Changes

Many updates to the Observation Table components and related components

Addressed a handful of dependency array warnings and "update a component while rendering another" warnings

Some misc package updates + related fixes.

Pending Critterbase updates

TODOs for another ticket

Testing Notes

codecov-commenter commented Feb 26, 2024 • edited Loading

Codecov Report

al-rosenthal left a comment • edited Loading

Choose a reason for hiding this comment

NickPhura commented Feb 27, 2024

al-rosenthal left a comment

Choose a reason for hiding this comment

NickPhura commented Feb 28, 2024

curtisupshall left a comment

Choose a reason for hiding this comment

al-rosenthal left a comment

Choose a reason for hiding this comment

al-rosenthal left a comment

Choose a reason for hiding this comment

sonarqubecloud bot commented Feb 29, 2024

Quality Gate failed

NickPhura commented Feb 14, 2024 •

edited

Loading

codecov-commenter commented Feb 26, 2024 •

edited

Loading

al-rosenthal left a comment •

edited

Loading